Method for converting the scanning format of images, a system and computer program product therefor

ABSTRACT

The conversion into a progressive format of digital images organized in half-frames or fields with interlaced lines or rows envisages selecting successive lines in one or more of said fields and reconstructing by pixels an image line set between the interlaced lines selected. The reconstruction operation obtains the image by creating a set of candidate patterns associated to the work window by selecting the patterns to be considered within the window. Next, applying to the patterns of the aforesaid set a first cost function which is representative of the correlations between pairs of pixels. Applying to the patterns of the aforesaid set a second cost function which is representative of the non-correlations between pairs of pixels. Selecting, for each candidate pattern, respective internal correlations and external non-correlations, calculating corresponding scores for the candidate patterns using the aforesaid first cost function. Selecting a best pattern by comparing the respective scores of the candidate patterns with at least one threshold; and selecting the pixels of the window identified by the best pattern selected, then reconstructing the missing line by filtration starting from said pixels.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to techniques that enable the conversionof the scanning format of digital images, in particular converting froman interlaced format to a progressive format.

2. Description of the Related Art

The technological limits that accompanied the original development oftelevision transmissions imposed the need for transmission of images in“interlaced” format, according to which the pixels of the odd and evenlines (also referred to as top and bottom lines) of an image are scannedand transmitted alternately.

The majority of the visual display devices currently existing (inparticular, cathode-ray-tube television apparatuses) support this typeof scanning for the incoming video signal. The refresh frequency and thepersistence of the image on the human eye for these apparatuses is infact such that the alternation of the odd and even lines is notperceived by the human eye. Consequently, the television standards usedpractically throughout the world (PAL, SECAM, and NTSC) adopt in factthe interlaced format.

The appearance of new technologies of construction of visual displayswhich enable screen dimensions to be obtained that are larger thancurrent ones (for instance plasma displays can reach 40″) has led to anincrease in the quality of the information being displayed andaccordingly an increase in the number of functions for processing thevideo signal.

Among the above functions one is precisely the conversion of scanning ofthe image from the interlaced format to the progressive format, whichrepresents the ideal type of image for exploiting to the full thecharacteristics of the new visual displays.

Consequently, various categories of de-interlacing solutions have beendeveloped, as evidenced, for example, by the following documents:

Pasi Pohjala, Matti Karlsson: Line rate upconversion in IDTVapplications, IEEE Transactions on Consumer Electronics, Vol. 37, No. 3,pp. 309–312, August 1991;

Kai Oistamo, Yrjo Neuvo: A motion insensitive method for scan rateconversion and cross error cancellation, IEEE Transactions on ConsumerElectronics, Vol. 37, No. 3, pp. 396–302, August 1991;

R. Simonetti, A. Polo Filisan, S. Carrato, G. Ramponi, G. Sicuranza: Adeinterlacer for IQTV receivers and multimedia applications, IEEETransactions on Consumer Electronics, Vol. 39, No. 3, pp. 234–240,August 1993;

H. Hwang, M. H. Lee, D. I. Song: Interlaced to progressive ScanConversion with double smoothing, IEEE Transactions on ConsumerElectronics, Vol. 39, No. 3, pp. 241–246, August 1993;

Dong-Ho Lee, Jong-Seok Park, Yung-Gil Kim: Video format conversionbetween HDTV Systems, IEEE Transactions on Consumer Electronics, Vol.39, No. 3, pp. 219–224, August 1993;

Rob A. Beukeir, Imran A. Shah: Analysis of interlaced video signals andits applications, IEEE Transactions on Image Processing, Vol. 3, No. 5,pp. 501–512, September 1994;

P. Delogne, Laurent Cuvelier, Benoit Maison, Beatrice Van Caillie, LucVanderdorpe: Improved interpolation, motion estimation, and compensationfor interlaced pictures, IEEE Transactions on Image Processing, Vol. 3,No. 5, pp. 482–491, September 1994;

Manfred Ernst: Motion compensated interpolation for advanced conversionand noise reduction, Proceedings 4th International Workshop on HDTV(Turin, Italy), Vol. 1, September 1991;

L. Capodiferro, A. Chiari, G. Marcone, S. Miceli: A screen formatconverter for HDTV, Proceedings 4th International Workshop on HDTV(Turin, Italy), Vol. 2, September 1991;

Bede Liu, Andre Zaccarin: A comparative study of coding of interlacedsequences with motion compensation, Proceedings 4th InternationalWorkshop on HDTV (Turin, Italy), Vol. 2, September 1991;

H. Blume, L. Schwoerer, K. Zygis: Subband based upconversion usingcomplementary median filters, Proceedings International Workshop on HDTV(Turin, Italy), October 1994;

Dong Wook Kang: Two-channel spatial interpolation of images, ImageCommunication 16, pp. 395–399, 2000; and

Feng-Ming Wang, Dimitris Anastassiou, Arun Netravali: Time-recursivedeinterlacing for IDTV and Pyramid Coding, Image Communication 2, pp.365–374, 1990.

With a certain degree of simplification, but with substantial adherenceto the actual situation, the above previously known solutions fit intofour basic categories, namely:

-   -   spatial techniques;    -   temporal techniques;    -   spatial-temporal techniques; and    -   motion-compensated techniques.

The various solutions differ a great deal both in terms of complexityand in terms of hardware resources required.

Spatial techniques use only the data coming from the same half-frame orfield. It is recalled that in the technical terminology of the sectorthe term ‘frame’ is used to indicate the entire image, whilst the terms‘half-frame’ or ‘field’ are used to indicate the interlaced image madeup of just odd lines or even lines. A frame is thus made up of twofields, referred to as top field and bottom field. In progressivescanning, both of these field are transmitted together, whereas ininterlaced scanning the two fields are transmitted alternately.

Temporal techniques use data coming from one or more fields, which aresubsequent and/or previous to the current field in which the missingdata to be reconstructed reside.

Spatial-temporal techniques use both types of data, i.e., both spatialones and temporal ones.

Finally, motion-compensated techniques are based on bothspatial-temporal data and on the estimation of the movement of thevarious parts of which the image is made up (calculation of the motionvectors). This motion information can be calculated externally and canhence be supplied from outside the system in question, or else may beestimated inside the conversion system.

In general terms, a greater number of available data enables a higherlevel of performance to be achieved. However, it may also happen that anincorrect use of this amount of data (for example, in the case oftechniques based upon motion compensation, an incorrect estimation ofthe motion-field values) leads the system to diverge from the idealsolution, introducing artifacts such as to degrade the quality of thereconstructed image.

BRIEF SUMMARY OF THE INVENTION

An embodiment provides an improved process for converting the scanningformat of images, and in particular for converting the scanning formatof digital images from an interlaced format to a progressive format,such as to overcome the typical drawbacks of the solutions present inthe literature.

According to an embodiment of the present invention, the above purposeis achieved thanks to a process for scanning conversion which has thecharacteristics referred to specifically in the claims which follow. Theinvention also relates to the corresponding system and computer-programproduct which, once loaded into the memory of a computer and runthereon, enables the process according to the invention to be carriedout.

In an embodiment according to the invention, the process makes itpossible to take into consideration the ideal factors for itsimplementation on a digital circuit, namely, the factors regarding thecomputational complexity necessary, the power consumption of the systemin the perspective of low-power applications, as well as theminimization of the requirements at a hardware level, the foregoingbeing combined, at the same time, with the search for, and achievementof, the best possible quantitative and qualitative results.

In addition, an embodiment of the process according to the inventiondoes not formulate any hypothesis on the type of processing carried outon the input information before it is de-interlaced. This means simplythat the function expressed is valid, in general, in a way irrespectiveof any possible encodings, decodings, treatment for reduction of noise,improvement of contrast and/or brightness and of any processing carriedout on the television signal.

As regards the amount of storage area occupied, and hence the memorynecessary for processing, it should be considered that a work windowthat is too small entails a reduced capacity for determining the objectsand hence a poor reconstruction of the missing information. On the otherhand, a work window that is too large may prove to be inefficient,oversized and hence unfavorable for the purposes of the complexity ofthe system.

In the spatial domain, the memory required for the work window isdirectly dependent upon the number N of rows or lines (the two terms“row” and “line” are used in an altogether equivalent way) of which thewindow is made up. In fact, assuming that the pixels that form the imageenter the system in raster mode, also referred to as sequential mode(consider as the main field of application the real-time processing oftelevision images), N lines for the work window correspond to N−1 delaylines-required, i.e., to N−1 line memories, plus the current line atinput to the system.

In the time domain, each line making up the aforesaid work window maybelong to images that are distant from one another in time. Each delayimage (i.e., each additional field in the work window) involves completestorage thereof; in this case, memories are required of a size that canbe a limitation on the practical implementation of a display device.

It should, in fact, be noted that between a line memory and a fieldmemory there is a factor of proportionality equal to the number of linesper field (which, in the PAL standard, is 288).

As pointed out previously, there exist algorithms based on motioncompensation. These may be divided into two distinct subclasses: withand without estimation of motion inside the system.

In the former case, it is necessary to have at least one image storedand the estimator of motion. In the latter case, even though the addedcomplication of the estimator is not present, there is no control overthe correctness of the motion data coming from outside, and it istherefore necessary to introduce functions of verification dedicated tothis purpose, with a consequent increase in the overall complexity ofthe system.

As regards power consumption, this factor is directly linked to thecomputational complexity of the solution adopted, or, in other words, tothe number of calculations made on average for processing. In general,the fundamental processing operations for the algorithms of the typedescribed above are the measurement of the correlations between pixels,the comparison between the correlations themselves in order to determinethe best solution, and, finally, the function of reconstruction of themissing information by means of specific filtering operators.

An embodiment according to the invention is thus based on the possibleextension to patterns that are different from rectangles andparallelograms, namely to patterns that are trapezoidal or triangular.The invention enables a proper equilibrium to be achieved between thecharacteristics so far described, in such a way as to be able to definean optimal hardware configuration according to the qualitative resultsattained and thus render the invention easily integrable in televisionapplications.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The present invention will now be described, purely by way ofnon-limiting example, with reference to the attached drawings, in which:

FIG. 1 illustrates the field structure processed using the processaccording to the invention;

FIG. 2 is a general illustration of the structure of the work window ofthe solution according to the invention;

FIG. 3 illustrates the concept of maximum and minimum slope in the workwindow of the solution according to the invention;

FIG. 4 shows four macro models of the directions of analysis and thecorresponding areas of pertinence of the solution according to theinvention;

FIG. 5 is a schematic illustration of the flow of the process ofde-interlacing of the video signal of the solution according to theinvention;

FIG. 6 illustrates in particular the internal flow of the search blockfor seeking the best solution using the solution according to theinvention; and

FIG. 7 illustrates, in the form of a block diagram, the structure of theentire processing model of the solution according to the invention.

DETAILED DESCRIPTION OF THE INVENTION

An embodiment according to the invention envisages carrying out theconversion of the scanning format of a digital video signal from aninterlaced format to a progressive format, using linear and non-linearfiltering techniques. The above process is performed on the basis ofpolygonal recognition of the objects and/or of their neighborhood.

In FIG. 1, the reference F designates, as a whole, a video-signal field,the lines of which are designated generically by pos(j), with j=1, . . ., n, where the missing lines, i.e., the ones on which the pixels to bereconstructed lie, are in general designated by pos(u+2k+1).

The working area of FIG. 2 is made up of a window W defined by portionsPZ consisting of R consecutive pixels belonging to D consecutive videolines. Comprised among these video lines is the missing line ML wherethe pixel of the image that is to be reconstructed lies.

In particular, in FIG. 2 the lines of the current field are designatedby pos(f+2R), whilst pos(f+2R+1) designates the lines of theprevious/next field/fields which may be motion dependent/non-dependent.

It is to be recalled that no hypothesis is formulated on the position ofthe portions PZ selected, on the balancing and on the number of pixelsR, or on the video lines D selected either. By the term “balancing” ismeant the selection of data in such a way as to consider the point orpixel to be reconstructed at the exact center of the work window.

As will emerge clearly from what follows, the choice illustrated is notto be considered imperative for the purposes of the implementation ofthe invention.

On the working area identified by the window W illustrated in FIG. 2, ameasurement of correlation between the pixels is determined in order tounderstand whether the latter belong or not to one and the same objectof the scene under examination. The object or portion thereof inside thewindow W will be hereinafter also referred to as pattern P.

In order to select the best pattern P corresponding to the objectpresent in the window W, a merit parameter is calculated, referred to as“score” G, which takes into account the correlations between the pixelspresent in the work window W.

The correlation between pairs of pixels is measured by calculating afunction, referred to as cost function F. The cost function that isapplied depends on a degree of correlation or non-correlation betweenthe pairs of pixels.

The number of the pairs of pixels depends upon the size of the window Wand the number of patterns P identified within the window.

The pairs are then used for calculation of the score G of each of theaforesaid patterns P identified.

Calculation of the score G is based upon two fundamental points:analysis of directional correlation and analysis of whether they belongto an object (pattern).

The best score G is then used to identify the pixels R that are to beused for reconstructing the missing datum. Also identified is themodality with which said pixels R are to be processed in order to obtainthe best result in the final filtering process.

In order to increase the robustness of the process in the event of anerror, for example in presence of noise, details, uncertain decisions,etc., there is preferably adopted a modality of flow of recognition ofthe best pattern by means of a targeted use of certain threshold valuesS, in order to discriminate as well as possible the values of the scoresG. The discrimination is performed on the basis of intrinsiccharacteristics of the patterns P, which will be clarified in whatfollows.

In brief, the de-interlacing technique according to the inventionenvisages carrying-out of the following macro-steps:

creation of the work window W, with selection of R points of the D lines(or rows);

creation of all the possible rectangular and polygonal patterns P withinthe aforesaid window W;

selection of the patterns P to be considered within the window W;

choice of the cost function F for measurement of the correlationsbetween pairs of pixels;

choice of the cost function FN for measurement of the non-correlationsbetween pairs of pixels;

for each pattern, choice of the number and type of internal correlationsand external non-correlations to be calculated;

calculation of the scores G for all the candidate patterns previouslyselected, using the cost function F previously chosen;

for each pattern, choice of the threshold values S with which the scorein the search flow for seeking the “best pattern” is to be compared;

search for the best candidate pattern by an appropriate search flow, andtargeted use of threshold values S;

on the basis of the best pattern previously found, choice of the pixelsR of the window W that are to form part of the final filtering process;

on the basis of the best pattern, choice of the best modality accordingto which said pixels are to be considered and used in the filteringprocess; and

application of the interpolation filter.

At this point, the missing information is reconstructed and can betransmitted and/or stored.

The transmission operation must respect the order and alternation withwhich the D lines were received at input to the de-interlacing system.

As was said previously, the process of identification of the points thatbest represent the pixel to be reconstructed is based upon theevaluation of the correlation between the pixels themselves, which ismeasured by means of the cost function F. It is essential to note thatthe choice of the cost function F affects the complexity and finalquality of the system itself.

FIG. 3 represents a work (or search) window W comprising four lines R1,R2, R3 and R4, between which there lies (in a central position) thepoint or pixel M to be reconstructed. Again in FIG. 3, L1 and L2designate two lines, respectively of minimum slope and maximum slope forthe purposes of determination of the correlation. The representation ofFIG. 3 is, in any case, provided purely by way of example, in particularas regards the number of lines represented.

The patterns used, denoted by P, are patterns of a rectangular type anda polygonal type, with the aim of considering the possible patterns ofdifferent thickness inside the window W. By then discriminating only thepatterns passing through the point of the pixel to be reconstructed, theone that best enables said reconstruction without introducing artifactsinto the image is selected.

It will be appreciated in particular that the patterns P present genericareas which depend upon the generic diagonal direction used for definingsaid patterns.

For a correct identification of the patterns, the dual search for thecorrelation and non-correlation of the pixels inside and outside thepattern P under examination is considered. From the above analyses itwill then be possible to identify the subset of the points within thewindow W that will form the basis for the subsequent and final filteringprocess.

The various patterns are then classified according to appropriate scoresG. These scores are then verified by means of appropriate selectioncriteria, which are essentially based upon the characteristicsthemselves of each pattern examined. An index, referred to as “index ofthe directionality of the pattern” has moreover been added foridentification of the types of patterns. The index of the directionalityidentifies the main direction of the diagonal component of each patternP.

In the case of the example of polygonal patterns of triangular shapeshown in FIG. 4, the directionality index is identified by the diagonalof the triangle itself.

The above example constitutes a specific case and is in no wayrestrictive of the altogether general nature of the system.

Again with reference to FIG. 4, the inclined side of the triangularpolygon has the function of indicating the direction of correlation,whilst the square areas underlying and overlying the diagonal indicatethe areas where the region to which the pixel to be reconstructed mightbelong is sought.

Patterns of a rectangular type have a fixed directionality index of 90degrees and are used for the search for uniform or non-uniform regionsin the neighborhood of the pixel to be reconstructed.

Patterns of a linear type are used for the identification of lineshaving a thickness of 1 pixel and passing through the pixel to bereconstructed (particular case of the types of previous patterns).

Once again with reference to the example of FIG. 4, it may be noted thatin the window W two main directionality indices X1 and X2, andconsequently four main areas of pertinence A, B, C and D, areidentified.

In what follows (in particular, with reference to the example of FIG.6), a description will be provided also of the use of the variousthresholds S with which first the best patterns P for each area ofpertinence are discriminated, and then compared one by one to obtain thebest pattern P for each direction of pertinence, and finally to identifythe best pattern in absolute terms in the window W.

The experiments conducted by the applicant show that the above solutionreduces the cases of erroneous recognition of the pattern P, and hencealso reduces the artifacts that may be introduced.

There have moreover been identified various sets of thresholds S, thevalues and uses of which depend directly upon their use within thesearch flow for seeking the best solution. In particular, three mainfamilies of thresholds S have been identified:

Threshold for Each Pattern P

Each score G calculated is compared with a threshold S in such a way asto eliminate the patterns that have a high degree of uncertainty. Inaddition, according to the specific type of pattern P, differentthreshold values are used to prevent penalizing the less likelypatterns. For example, patterns with a very gentle diagonal slope proveintrinsically less likely, and hence could prove, to be constantlypenalized if compared with patterns having a steep slope.

Thresholds for Selection of the Patterns Belonging to One and the SameDirectionality Index

These are used for discriminating the cases of specular ambiguity of twoopposite areas along the same direction.

Thresholds for Selection of the Patterns Belonging to OppositeDirectionalities

These are used to solve the cases of ambiguity in the event of oppositedirections.

The operating flow defined by the proposed solution defines theprogressive selection of the winning patterns P along the various casesdefined previously.

In general, the aforesaid thresholds S are used in order to solve thecases of ambiguity in the event of high degrees of correlation betweentwo or more scores of the patterns under examination (equiprobablepatterns).

In the comparison between two patterns, the function of correlationbetween the two corresponding scores is compared with the threshold. Inthe case where the two patterns prove equiprobable it is necessary toadopt an adequate methodology to solve the ambiguity. The methodologymay consist in eliminating the two candidate patterns from the algorithmfor choice of the best pattern, or else in using a default functionchosen beforehand. The choice is linked to the difference between thetwo candidate patterns, and hence to the criticality of a possibleerroneous choice, which may lead, for example, to a polarization of theinterpolation in a certain direction.

The threshold values S are not fixed a priori, but may be defined evenduring application of the system.

The flowchart of FIG. 5 illustrates, in greater detail, the developmentof the operating sequence upon which the operation of the solutionaccording to the invention is based.

Starting from a start step 100, the process envisages the construction(according to the modalities described previously) of the work window W.This operation is schematically represented by the step 101.

Next, all the possible patterns inside the work window are constructed(step 102), and this is followed by a step of selection (step 103) ofthose patterns alone that are related to the central pixel of the workwindow W, i.e., the pixel to be reconstructed.

At the same time, the correlation function (step 104) andnon-correlation function (step 105) are selected, which will serve forthe computation of the scores G of the patterns selected in step 103.

At this point, there are available all the elements necessary for theactual calculation of the score (step 106) of each candidate pattern. Inthis step there are in fact considered the correlation between thepoints inside the pattern in question, the correlation between thepixels outside the pattern, and finally the non-correlation between thepixels inside and outside the pattern.

In order to improve the identification of an area of pertinence of thedatum to be reconstructed, the aforesaid correlations are calculated asa function of the correlations calculated along the perimeter of thepattern under examination, as a function of the correlations calculatedin the area of the pattern under examination, as a function of thecorrelations calculated outside the perimeter of the pattern inquestion, and finally as a function of the non-correlations in thedirection inside-outside the pattern.

The value of the score associated to the pattern is thus a function(step 107) of the analyses of correlation, both internal and external,described previously, in such a way as to identify uniquely the patternwith its corresponding score.

The above step is directly connected to the use of certain thresholdvalues corresponding to specific patterns, so as to even out theprobability of selection for each pattern, as explained previously. Thestep (step 112) hence establishes, according to the selection of thepatterns made during step 103, the threshold values for the patternsthus defined.

At the end of the calculation carried out in step 107, there areavailable all the scores necessary for the search for the best solutionwithin the window W. This search step (step 108) constitutes the centralpart of the process for determining the best pattern, and it is on thisstep that there depends the quality of the subsequent process ofselection of the most suitable pixels, and hence also the way in whichthey are to be processed in order to reconstruct the missing informationin the best possible way. Step 108 will be analyzed in further detaillater on, at the end of the description of the present work flow (FIG.6).

At the end of step 108, it is thus possible to proceed simultaneously,in step 109 and step 110, respectively to the selection of the pixelsthat depend upon the best pattern previously calculated, and to thechoice of the function that is to be applied to the pixels for the finalfiltering process.

As has been said, the final step 111 consists precisely in the operationof filtering the pixels selected, according to the modality selected.The modality may be carried out by using linear and nonlinear functions,according to the particular type of pattern or to the implementationalchoices made.

Once step 111 is through, the process may be repeated iteratively inorder to reconstruct the remaining pixels that are missing within theimage lines of the interlaced image, and proceeding to storage and/ordisplay of the pixels (step 113).

It is important to remember that each iteration of the process ofcalculation, analysis and reconstruction does not involve any assumptionon the results obtained in the previous iterations, so as not topropagate possible errors.

FIG. 6 provides a detailed illustration of the part of the system forsearch for the best solution of step 108 of FIG. 5 within the patterns Pselected inside the work window W.

The process consists in the parallel search for two basic types ofpatterns, namely, the so-called diagonal patterns and the so-calledrectangular patterns. As far as linear patterns are concerned, thesefall within the two cases mentioned above, in view of the fact that theymay be identified as polygonal and/or rectangular patterns having thethickness of 1 pixel.

In the branch of search for the best solution within the diagonalpatterns, four steps that can be processed in parallel may beidentified. The steps, 200, 201, 202 and 203, have the function ofcalculating the correlations and corresponding scores of the patternsbelonging to the four categories of diagonal patterns A, B, C and D, asrepresented schematically in FIG. 4. The four categories are preciselythose in which the two diagonal macro-directions establish fourmacro-areas where it is possible to construct the diagonal patterns.

In order to reduce the number of cases to be analyzed to the ones thathave the highest probability of being selected as winning solutions atthe end of the entire search process, each score for each pattern iscompared with an appropriate threshold value (A)—steps 204, 205, 206,and 207.

Within each of the four categories, it thus established which of thepatterns originally considered as likely is recognized with the highestdegree of certainty.

It is thus possible to establish the most significant pattern for eachof the four categories of patterns—final phase of the aforesaid steps204, 205, 206, and 207.

There will then be defined two representatives of the two differentdirectionalities defined previously (FIG. 4).

The cases of ambiguity that arise with a similar score among differentpatterns belonging to the same area are resolved by selecting the leastlikely pattern among the best ones, the recognition of which is thusrewarded.

Next, the winning patterns PV are analyzed again in order to improve theselection further. A function of analysis between pairs of patterns PVand a further threshold (B) are then applied. The winning patterns arepreviously grouped according to a criterion of pertinence to their owndirectionality.

Once again, the cases of ambiguity that occur with a high correlationbetween different patterns belonging to different areas—a valueindicated by the threshold (B)—are resolved by choosing a defaultsolution that operates a non-critical filtration. It is clear that thisdefault solution is chosen uniquely in the cases where in both of theflows—step 208 and step 209—the ambiguities occur.

There are then defined two representatives of the two directionalitiesdefined. These patterns are the winning patterns PV1 and PV2, eachconsisting of the winners in steps 204 and 205, on the one hand, and thewinners in steps 206 and 207, on the other. This step is goes by thename of specular analysis with respect to a diagonal direction.

The same applies to the next step 210, in which the winning patterns PVVare analyzed together and with an appropriate threshold (C). Thisverification process follows the same methodology as that defined in theprevious step. Again, in the case where more than one pattern prove tobe equally winning, in the present case it is advisable to opt for adefault solution chosen a priori, in such a way as to prevent apolarization of the selection process. This step goes by the name ofopposite directional analysis.

In the branch of search for the best solution within the rectangularpatterns, the scores of the patterns that have no diagonality arecalculated. This step (step 211) determines the best pattern in terms ofuniform regions present in the search window. This pattern is thenanalyzed together with an appropriate threshold (D) in order to certifyits actual probability of final selection.

The final step, verification of the best pattern, which concludes thesearch process for the best pattern (step 213) considers then thediagonal winning pattern, the vertical winning pattern, and a furthercontrol threshold (E). This verification process follows the samemethodology as that defined in the previous step. The output (214) ofthis step constitutes the winning pattern in absolute terms among allthe possible patterns within the work window W.

The solution so far described essentially corresponds to the search forthe minimum value among the correlations of the patterns identified, andhence to the identification of the pixels to be used in the finalfiltering process.

The flow proceeds with the determination of the pixels that will concurin the subsequent filtering process, and hence in the choice of themodality with which the latter is implemented.

For this purpose, selection rules are followed that depend upon thecharacteristics themselves of the winning pattern: patterns with a verygentle diagonality suggest the use of both pixels belonging to thepattern itself and pixels in the region outside the pattern; patternswith a very steep diagonality suggest the use of pixels belonging onlyto the pattern itself; intermediate degrees of diagonality entail theadoption of empirical rules for the selection of the pixels for thefinal filtering step.

As regards the subsequent filtering step, the solution according to theinvention envisages selection between linear and non-linear techniquesto be applied to the process. The choice of one technique or anotherdepends upon various factors, such as the type of winning patternselected, the number of pixels belonging to the aforesaid pattern whichare considered, and other factors that are strictly linked to thechoices for the implementation of the final system.

It has been determined that gentler slopes are indicative of a lowercorrelation between the pixels belonging to the diagonal of the pattern,and hence the choice of a non-linear function that tends to preserve thevalues of the original pixels intrinsically has a higher probability ofintroducing artifacts that are visible in the final image; vice versa,in the case of patterns with steeper slopes.

The block diagram of FIG. 7 illustrates in general the structure of aprocessing system designed to generate, starting from an input signal ISconsisting of a digital video signal in “interlaced” format (i.e., onein which pixels of the even lines and odd lines are transmittedalternately) into an output signal OS organized in a progressive format.The output signal OS can be displayed on an appropriate display, or elsecan be stored for further processing.

In the diagram of FIG. 7, the reference number 11 designates a storagebuffer, of a known type, that performs the function of line and fieldmemories. The data at output from this block can possibly depend uponmotion information.

The reference number 12 represents a block corresponding to amotion-compensation system comprising a motion-estimation engine 12 a,which, possibly co-operating with an external source ES, supplies theinput to a module 12 b for determination of the motion vectors. Theblock may supply the motion vectors received from outside, or elsecalculate them with an appropriate motion-estimation system andcorresponding connection to the input buffer.

The reference number 13 designates a block in which the work window isprovided on which the rules for search for the pattern, 14, 15 and 16,and for selection of the pixels, 17, for the final filtering operation18 are applied.

The reference number 14 designates the block for determining andselecting the patterns applicable within the window in step 13.

The reference number 15 designates the module for calculation of thecorrelations of the pixels of the window W and the function ofcomposition of the scores G of the patterns selected in step 14.

The reference number 16 designates the module for search for the bestpattern P within the work window W in order to reconstruct the missingdatum within the window W.

The reference number 17 designates the block for selection of the pixelsfor the subsequent filtering process. Said pixels will be selectedwithin the window W on the basis of the best pattern estimated in step16 and of the filtering modality.

The reference number 18 designates the final filtering module in whichthe datum missing in the window W is calculated exactly, and hence thefiltering module constitutes the final part of the de-interlacingengine. Subsequently, the progressive data constituting the outputsignal OS can be displayed on an appropriate progressive-scanning visualdisplay 19, or else can be stored in special buffers 20, for possiblefurther processing.

Of course, without prejudice to the principle of the invention, thedetails of construction and the embodiments may vary widely with respectto what is described and illustrated herein, without thereby departingfrom the scope of the present invention as defined in the annexedclaims.

1. A process for converting into progressive format digital imagesorganized in half-frames or fields with interlaced lines or rows,comprising the operations of: selecting a set of successive lines of oneof said fields; and reconstructing by pixels an image line set betweenthe successive lines of said set, the successive lines of said setdefining a work window, said reconstruction operation comprising thesteps of: creating a set of candidate patterns associated to said workwindow by selecting the candidate patterns to be considered within saidwork window; applying to the candidate patterns of said set of candidatepatterns a first cost function which is representative of thecorrelations between pairs of pixels; applying to the candidate patternsof said set of candidate patterns a second cost function which isrepresentative of the non-correlations between pairs of pixels;selecting, for each candidate pattern, respective internal correlationsand external non-correlations, calculating corresponding scores for saidcandidate patterns using said first cost function; selecting a bestpattern by comparing the corresponding scores of the candidate patternswith a threshold; and selecting the pixels of said work windowidentified by the best pattern selected, reconstructing said image lineby filtration starting from said pixels.
 2. The process according toclaim 1, wherein said score is based on an analysis of a directionalcorrelation and on an analysis of a pertinence to an object.
 3. Theprocess according to claim 1, wherein said patterns are polygonalpatterns in a framework of said work window.
 4. The process according toclaim 3, wherein said patterns are patterns of a rectangular ortriangular type.
 5. The process according to claim 3, wherein saidpatterns are patterns of a triangular type.
 6. The process according toclaim 3, wherein said patterns present a diagonal component, thediagonal component having a direction of which identifies acorresponding index of directionality.
 7. The process according to claim1, wherein an area of said candidate patterns is indicative of an objectof pertinence to a framework of said images.
 8. The process according toclaim 1, further comprising an operation of discriminating saidcandidate patterns by means of a threshold for countering an effect oferroneous recognition of the candidate patterns themselves withconsequent possible generation of artifacts.
 9. The process according toclaim 8, wherein said operation of applying the threshold includes theapplication of thresholds from the group consisting of: thresholds foreach candidate pattern by elimination of the candidate patterns with ahigh degree of uncertainty; thresholds for selection of the candidatepatterns having an index of directionality that is the same; andthresholds for selection of the candidate patterns having adirectionality being opposite.
 10. The process according to claim 1,further comprising an operation of subjecting the pixels of saidreconstructed image line to an interpolation operation.
 11. The processaccording to claim 1, wherein said corresponding scores for thecandidate patterns include: a correlation between the pixels inside thecandidate pattern; a correlation between the pixels outside saidcandidate pattern; and a non-correlation between the pixels inside andoutside said candidate pattern.
 12. The process according to claim 11,wherein said correlations are calculated as functions chosen in thegroup consisting of: correlations calculated along a perimeter of thecandidate pattern; correlations calculated in an area of the candidatepattern; correlations calculated outside the perimeter of the candidatepattern; and non-correlations in a direction inside-outside thecandidate pattern.
 13. A system for converting into progressive formatdigital images organized in half-frames with interlaced lines,comprising: a selection set for selecting a set of successive lines ofsaid field; and a processing set for reconstructing by pixels an imageline set between the successive lines of said set of successive lines ofsaid field; and said selection set is configured for: defining, startingfrom the successive lines of said set of successive lines, a work windowhaving a patterns; creating a set of candidate patterns associated tosaid work window by selecting the patterns to be considered within saidwork window; applying to the candidate patterns of said set of candidatepatterns a first cost function which is representative of thecorrelations between pairs of pixels; applying to the candidate patternsof said set of candidate patterns a second cost function which isrepresentative of the non-correlations between pairs of pixels;selecting, for each candidate pattern, respective internal correlationsand external non-correlations, calculating corresponding scores for saidcandidate patterns using said first cost function; selecting a bestpattern by comparing the corresponding scores of the candidate patternswith a threshold; and selecting the pixels of said work windowidentified by the best pattern selected, reconstructing said image lineby filtration starting from said pixels.
 14. The system according toclaim 13, wherein the processing set calculates said score by analysisof a directional correlation and analysis of a pertinence to an object.15. The system according to claim 14, wherein the selection setcomprises a memory for storing at least part of one of said successivelines of said selection set.
 16. The system according to claim 13,wherein said candidate patterns are polygonal patterns in a framework ofsaid work window.
 17. The system according to claim 16, wherein saidcandidate patterns are a rectangular or triangular type.
 18. The systemaccording to claim 16, wherein said candidate patterns are triangularpatterns.
 19. The system according to claim 16, wherein said candidatepatterns present a diagonal component, the diagonal component having adirection of which identifies a corresponding index of directionality.20. The system according to claim 13, wherein an area of said candidatepatterns is indicative of an object belonging in a framework of saidimages.
 21. The system according to claim 13, wherein said processingset is configured to discriminate said candidate patterns by means of athreshold for countering the effects of erroneous recognition of thecandidate patterns themselves with consequent possible generation ofartifacts.
 22. The system according to claim 21, wherein said processingset is configured for applying thresholds from the group consisting of:thresholds for each candidate pattern by elimination of the candidatepatterns with a high degree of uncertainty; thresholds for selection ofthe candidate patterns with the same index of directionality; andthresholds for selection of the candidate patterns having oppositedirectionalities.
 23. The system according to claim 13, wherein saidprocessing set is configured to subject the pixels of said reconstructedimage line to an interpolation operation.
 24. The system according toclaim 13, wherein said processing set calculates said correspondingscores for the candidate patterns, including: a correlation between thepixels inside the candidate pattern; a correlation between the pixelsoutside said candidate pattern; and a non-correlation between the pixelsinside and outside said candidate pattern.
 25. The system according toclaim 24, wherein the correlations are calculated as functions from thegroup consisting of: correlations calculated along a perimeter of thecandidate pattern; correlations calculated in an area of the candidatepattern; correlations calculated outside the perimeter of the candidatepattern; and non-correlations in the direction inside-outside thecandidate pattern.
 26. A computer program product directly loadable inthe memory of a digital computer comprising software code portions forperforming the following steps when said product is run on a computer:selecting a set of successive lines of one of said fields; andreconstructing by pixels an image line set between the successive linesof said set, the successive lines of said set defining a work window,said reconstruction operation comprising the steps of: creating a set ofcandidate patterns associated to said work window by selecting thecandidate patterns to be considered within said work window; applying tothe candidate patterns of said set of candidate patterns a first costfunction which is representative of the correlations between pairs ofpixels; applying to the candidate patterns of said set of candidatepatterns a second cost function which is representative of thenon-correlations between pairs of pixels; selecting, for each candidatepattern, respective internal correlations and external non-correlations,calculating corresponding scores for said candidate patterns using saidfirst cost function; selecting a best pattern by comparing thecorresponding scores of the candidate patterns with a threshold; andselecting the pixels of said work window identified by the best patternselected, reconstructing said image line by filtration starting fromsaid pixels.
 27. A process of converting an image comprising: selectinga plurality of lines of a first format wherein the plurality of lines isformed of a plurality of pixels of a first image; selecting a workwindow having a plurality of patterns wherein the plurality of patternsis formed of a subset of pixels of the plurality of pixels of the firstimage; determining a plurality of candidate patterns from the pluralityof patterns of the work window; determining correlations of pairs ofpixels of the work window to determine if the pairs of pixels areassociated with a selected pattern of the plurality of candidatepatterns; determining a score of the selected pattern; comparing athreshold to the score of the selected pattern to obtain a best pattern;and determining a target pixel of the subset of pixels of the pluralityof pixels of the best pattern wherein the target pixel forms a portionof a second image in a second format.
 28. The process according to claim27 wherein determining the correlations of the pairs of pixels includesapplying a cost function representative of a degree of the correlationto the selected pattern.
 29. The process according to claim 27 whereincomparing the threshold to the score further includes selecting a firstthreshold of selected patterns having a steep slope.
 30. The processaccording to claim 27 wherein comparing the threshold to the scorefurther includes selecting a second threshold of selected patternshaving first and second areas wherein the areas are opposite and in thesame direction.
 31. The process according to claim 27 wherein comparingthe threshold to the score further includes selecting a third thresholdof selected patterns having an opposite direction.
 32. The computerprogram product according to claim 26, wherein said score is based on ananalysis of a directional correlation and on an analysis of a pertinenceto an object.
 33. The computer program product according to claim 26,wherein said patterns are polygonal patterns in a framework of said workwindow.
 34. The computer program product according to claim 26, whereinsaid patterns present a diagonal component, the diagonal componenthaving a direction of which identifies a corresponding index ofdirectionality.
 35. The computer program product according to claim 26,further comprising software code portions for performing the followingwhen said product is run on a computer: discriminating said candidatepatterns by a threshold for countering an effect of erroneousrecognition of the candidate patterns themselves with consequentpossible generation of artifacts.
 36. The computer program productaccording to claim 35, wherein said discriminating said candidatepatterns by the threshold includes applying thresholds from a groupconsisting of: thresholds for each candidate pattern by elimination ofthe candidate patterns with a high degree of uncertainty; thresholds forselection of the candidate patterns having an index of directionalitythat is the same; and thresholds for selection of the candidate patternshaving a directionality being opposite.
 37. The computer program productaccording to claim 26, further comprising software code portions forperforming the following when said product is run on a computer:subjecting the pixels of said reconstructed image line to aninterpolation operation.
 38. The computer program product according toclaim 26, wherein said corresponding scores for the candidate patternsinclude: a correlation between the pixels inside the candidate pattern;a correlation between the pixels outside said candidate pattern; and anon-correlation between the pixels inside and outside said candidatepattern.
 39. The computer program product according to claim 39, whereinsaid correlations are calculated as functions chosen in the groupconsisting of: correlations calculated along a perimeter of thecandidate pattern; correlations calculated in an area of the candidatepattern; correlations calculated outside the perimeter of the candidatepattern; and non-correlations in a direction inside-outside thecandidate pattern.